Methods for linear radial motion estimation in time-of-flight range imaging

Motion artefacts in time-of-flight range imaging are treated as a feature to measure. Methods for measuring linear radial velocity from range imaging cameras are developed and tested. With the measurement of velocity, the range to the position of the target object at the start of the data acquisition period is computed, effectively correcting the motion error. A new phase based pseudo-quadrature method designed for low speed measurement measures radial velocity up to ±1.8 m/s with RMSE 0.045 m/s and standard deviation of 0.09-0.33 m/s, and new high-speed Doppler extraction method measures radial velocity up to ±40 m/s with standard deviation better than 1 m/s and RMSE of 3.5 m/s

Source Modulated Multiplexed Hyperspectral Imaging: Theory, Hardware and Application

The design, analysis and application of a multiplexing hyperspectral imager is presented. The hyperspectral imager consists of a broadband digital light projector that uses a digital micromirror array as the optical engine to project light patterns onto a sample object. A single point spectrometer measures light that is reflected from the sample. Multiplexing patterns encode the spectral response from the sample, where each spectrum taken is the sum of a set of spectral responses from a number of pixels. Decoding in software recovers the spectral response of each pixel. A technique, which we call complement encoding, is introduced for the removal of background light effects. Complement encoding requires the use of multiplexing matrices with positive and negative entries. The theory of multiplexing using the Hadamard matrices is developed. Results from prior art are incorporated into a singular notational system under which the different Hadamard matrices are compared with each other and with acquisition of data without multiplexing (pointwise acquisition). The link between Hadamard matrices with strongly regular graphs is extended to incorporate all three types of Hadamard matrices. The effect of the number of measurements used in compressed sensing on measurement precision is derived by inference using results concerning the eigenvalues of large random matrices. The literature shows that more measurements increases accuracy of reconstruction. In contrast we find that more measurement reduces precision, so there is a tradeoff between precision and accuracy. The effect of error in the reference on the Wilcoxon statistic is derived. Reference error reduces the estimate of the Wilcoxon, however given an estimate of theWilcoxon and the proportion of error in the reference, we show thatWilcoxon without error can be estimated. Imaging of simple objects and signal to noise ratio (SNR) experiments are used to test the hyperspectral imager. The simple objects allow us to see that the imager produces sensible spectra. The experiments involve looking at the SNR itself and the SNR boost, that is ratio of the SNR from multiplexing to the SNR from pointwise acquisition. The SNR boost varies dramatically across the spectral domain from 3 to the theoretical maximum of 16. The range of boost values is due to the relative Poisson to additive noise variance changing over the spectral domain, an effect that is due to the light bulb output and detector sensitivity not being flat over the spectral domain. It is shown that the SNR boost is least where the SNR is high and is greatest where the SNR is least, so the boost is provided where it is needed most. The varying SNR boost is interpreted as a preferential boost, that is useful when the dominant noise source is indeterminate or varying. Compressed sensing precision is compared with the accuracy in reconstruction and with the precision in Hadamard multiplexing. A tradeoff is observed between accuracy and precision as the number of measurements increases. Generally Hadamard multiplexing is found to be superior to compressed sensing, but compressed sensing is considered suitable when shortened data acquisition time is important and poorer data quality is acceptable. To further show the use of the hyperspectral imager, volumetric mapping and analysis of beef m. longissimus dorsi are performed. Hyperspectral images are taken of successive slices down the length of the muscle. Classification of the spectra according to visible content as lean or nonlean is trialled, resulting in a Wilcoxon value greater than 0.95, indicating very strong classification power. Analysis of the variation in the spectra down the length of the muscles is performed using variography. The variation in spectra of a muscle is small but increases with distance, and there is a periodic effect possibly due to water seepage from where connective tissue is removed from the meat while cutting from the carcass. The spectra are compared to parameters concerning the rate and value of meat bloom (change of colour post slicing), pH and tenderometry reading (shear force). Mixed results for prediction of blooming parameters are obtained, pH shows strong correlation (R² = 0.797) with the spectral band 598-949 nm despite the narrow range of pH readings obtained. A likewise narrow range of tenderometry readings resulted in no useful correlation with the spectra. Overall the spatial multiplexed imaging with a DMA based light modulation is successful. The theoretical analysis of multiplexing gives a general description of the system performance, particularly for multiplexing with the Hadamard matrices. Experiments show that the Hadamard multiplexing technique improves the SNR of spectra taken over pointwise imaging. Aspects of the theoretical analysis are demonstrated. Hyperspectral images are acquired and analysed that demonstrate that the spectra acquired are sensible and useful

Colour image processing and texture analysis on images of porterhouse steak meat

This paper outlines two colour image processing and texture analysis techniques applied to meat images and assessment of error due to the use of JPEG compression at image capture. JPEG error analysis was performed by capturing TIFF and JPEG images, then calculating the RMS difference and applying a calibration between block boundary features and subjective visual JPEG scores. Both scores indicated high JPEG quality. Correction of JPEG blocking error was trialled and found to produce minimal improvement in the RMS difference. The texture analysis methods used were singular value decomposition over pixel blocks and complex cell analysis. The block singular values were classified as meat or non- meat by Fisher linear discriminant analysis with the colour image processing result used as ‘truth.’ Using receiver operator characteristic (ROC) analysis, an area under the ROC curve of 0.996 was obtained, demonstrating good correspondence between the colour image processing and the singular values. The complex cell analysis indicated a ‘texture angle’ expected from human inspection

Optical full Hadamard matrix multiplexing and noise effects: errata

The model for Poisson random noise under Hadamard multiplexing is revised. The new model accounts for the variation of the Hadamard multiplexed measurements, as well as the previously considered variation due to Poisson fluctuations. A numerical simulation matches the model prediction within uncertainty

Optical full Hadamard matrix multiplexing and noise effects

Hadamard multiplexing provides a considerable SNR boost over additive random noise but Poisson noise such as photon noise reduces the boost. We develop the theory for full H-matrix Hadamard transform imaging under additive and Poisson noise effects. We show that H-matrix encoding results in no effect on average on the noise level due to Poisson noise sources while preferentially reducing additive noise. We use this result to explain the wavelength-dependent varying SNR boost in a Hadamard hyperspectral imager and argue that such a preferential boost is useful when the main noise source is indeterminant or varying

Separating true range measurements from multi-path and scattering interference in commercial range cameras

Time-of-flight range cameras acquire a three-dimensional image of a scene simultaneously for all pixels from a single viewing location. Attempts to use range cameras for metrology applications have been hampered by the multi-path problem, which causes range distortions when stray light interferes with the range measurement in a given pixel. Correcting multi-path distortions by post-processing the three-dimensional measurement data has been investigated, but enjoys limited success because the interference is highly scene dependent. An alternative approach based on separating the strongest and weaker sources of light returned to each pixel, prior to range decoding, is more successful, but has only been demonstrated on custom built range cameras, and has not been suitable for general metrology applications. In this paper we demonstrate an algorithm applied to both the Mesa Imaging SR-4000 and Canesta Inc. XZ-422 Demonstrator unmodified off-the-shelf range cameras. Additional raw images are acquired and processed using an optimization approach, rather than relying on the processing provided by the manufacturer, to determine the individual component returns in each pixel. Substantial improvements in accuracy are observed, especially in the darker regions of the scene

Time-of-flight range image measurement in the presence of transverse motion using the Kalman filter

Time-of-flight range imaging cameras measure distance to objects in their field of view, but are prone to error when objects move. At least three raw frames are required to obtain one range image, and the standard method is to read out raw frames into separate sets and process to find one range image per set. Motion during the acquisition of a set causes error in the corresponding range image. In this paper, the problem of motion is addressed by regarding the raw data from each pixel as a noisy time series, and using the Kalman filter to efficiently perform time-series analysis. The proposed method adapts to the effects of transverse motion, measuring a sharp range image at each raw frame. The error in the proposed method is less than the traditional approach in 80% of tests, with no detected increase in the STD due to noise. In the qualitative experimental results, the visible blur is reduced

Extracting the MESA SR4000 Calibrations

Time-of-flight range imaging cameras are capable of acquiring depth images of a scene. Some algorithms require these cameras to be run in `raw mode', where any calibrations from the off-the-shelf manufacturers are lost. The calibration of the MESA SR4000 is herein investigated, with an attempt to reconstruct the full calibration. Possession of the factory calibration enables calibrated data to be acquired and manipulated even in “raw mode.” This work is motivated by the problem of motion correction, in which the calibration must be separated into component parts to be applied at different stages in the algorithm. There are also other applications, in which multiple frequencies are required, such as multipath interference correction. The other frequencies can be calibrated in a similar way, using the factory calibration as a base. A novel technique for capturing the calibration data is described; a retro-reflector is used on a moving platform, which acts as a point source at a distance, resulting in planar waves on the sensor. A number of calibrations are retrieved from the camera, and are then modelled and compared to the factory calibration. When comparing the factory calibration to both the “raw mode” data, and the calibration described herein, a root mean squared error improvement of 51:3mm was seen, with a standard deviation improvement of 34:9mm. © (2015) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only

Classifying Transverse Motion in Time-of-Flight Range Imaging

Classification of step motion in time-of-flight imaging using the stochastic oscillator and autocorrelation is proposed. Machine learning algorithms correctly identify the step location in 65–75% of trials, with apparent good noise robustness

Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles

Time of flight cameras produce real-time range maps at a relatively low cost using continuous wave amplitude modulation and demodulation. However, they are geared to measure range (or phase) for a single reflected bounce of light and suffer from systematic errors due to multipath interference. We re-purpose the conventional time of flight device for a new goal: to recover per-pixel sparse time profiles expressed as a sequence of impulses. With this modification, we show that we can not only address multipath interference but also enable new applications such as recovering depth of near-transparent surfaces, looking through diffusers and creating time-profile movies of sweeping light. Our key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns. Applications to computer vision include ranging of near-transparent objects and subsurface imaging through diffusers. Our low cost prototype may lead to new insights regarding forward and inverse problems in light transport.United States. Defense Advanced Research Projects Agency (DARPA Young Faculty Award)Alfred P. Sloan Foundation (Fellowship)Massachusetts Institute of Technology. Media Laboratory. Camera Culture Grou